Skip to main content

DDL

Overview​

info

This document is for TimeBase Community Edition and TimeBase Enterprise Edition 5.5.

The TimeBase Data Definition Language (DDL) allows users to create, drop, and modify streams in TimeBase by executing QQL statements, similar to CREATE TABLE ... in SQL.

In QQL, the equivalent statement begins with CREATE ... STREAM, and thus, is called the Create Stream statement.

Create a Stream​

You can create two types of streams:

Syntax​

The following grammar is used to create a stream:

CREATE TRANSIENT|DURABLE STREAM [IF NOT EXISTS] stream_name [title]
(class_expression|enum_expression [; ...])
[OPTIONS (identifier [= expr] [; ...])]
[COMMENT 'comment text']

class_expression is described as follows:  

CLASS type_name [title] [UNDER type_name]
(static_attribute|attribute [, ...])
[AUXILIARY|NOT AUXILIARY]
[INSTANTIABLE|NOT INSTANTIABLE]
[COMMENT 'comment text']

enum_expression is described as follows:  

ENUM enum_name [title]
(identifier [= expr] [, ...])
[FLAGS]
[COMMENT 'comment text']

static_attribute has the following grammar:  

STATIC identifier [title] type [NOT NULL] [encoding] [BETWEEN min_expr AND max_expr] = expr
[TAGS (identifier:expr [, ...])]
[COMMENT 'comment text']

attribute has the following grammar:  

identifier [title] type [NOT NULL] [encoding] [BETWEEN min_expr AND max_expr] [RELATIVE TO identifier] [DEFAULT expr]
[TAGS (identifier:expr [, ...])]
[COMMENT 'comment text']

Options​

The OPTIONS list below contains all options for identifier:

Click to see stream `OPTIONS` attributes.
  • FIXEDTYPE: (Boolean.) A stream capable of containing messages of a single specified type.
  • POLYMORPHIC: (Boolean.) A polymorphic stream is capable of containing messages of several specified types.
  • LOSSLESS: (Boolean.) Lossless stream. Durable streams are always lossless.
  • LOSSY: (Boolean.) Lossy stream.
  • HIGHAVAILABILITY: (Boolean.) High availability durable streams are cached on startup.
  • PERIODICITY: (Varchar.) Indicate a known stream periodicity.
  • DF: (Numeric.) Distribution factor value.
  • INITSIZE: (Numeric.) Initial size of the write buffer in bytes.
  • MAXSIZE: (Numeric.) The limit on buffer growth in bytes. Default is 64K.
  • MAXTIME: (Numeric.) The limit on buffer growth as difference between first and last message time. Default is Long.MAX_VALUE.

Identifier​

An identifier is a token that forms a name.

Tips​

  • Use UNDER to indicate a parent class if applicable.
  • Classes may include static and/or non-static attributes. You can list them separated by a comma.
  • AUXILIARY classes cannot be written to a stream, but stream objects may contain AUXILIARY classes. NOT AUXILIARY classes can be written to a stream.
  • NOT INSTANTIABLE are abstract classes.
  • Use FLAGS to store enum values encoded as bitmask.
  • Supply a specific expr values with STATIC attributes.
  • Use BETWEEN/END to assign minimal and maximal values to numeric attributes.
  • Non-static attributes may acquire DEFAULT values.
  • Use RELATIVE TO to indicate that a non-static attribute's decoding depends on another field/property value.
  • With TAGS you can store a specific field-related metadata as a key:value pair.
  • Use COMMENT to add your message to the query.

For a general overview, refer to the QQL Introduction page.

Example​

CREATE DURABLE STREAM TEST (
    CLASS "deltix.timebase.api.messages.MarketMessage" 'Market Message' (
        "currencyCode" 'Currency Code' INTEGER SIGNED (16) COMMENT 'Currency code represented as short',
        "originalTimestamp" 'Original Timestamp' TIMESTAMP COMMENT 'Exchange Time is measured in milliseconds that passed since January 1, 1970 UTC',
        "sequenceNumber" 'Sequence Number' INTEGER COMMENT 'Market specific identifier of the given event in a sequence of market events',
        "sourceId" 'Source Id' VARCHAR ALPHANUMERIC (10) COMMENT 'Identifies market data source' 
    ) AUXILIARY;
      
    CLASS "deltix.timebase.api.messages.BestBidOfferMessage" 'Quote Message' UNDER "deltix.timebase.api.messages.MarketMessage" (
        "offerPrice" 'Offer Price' FLOAT DECIMAL (2),
        "offerSize" 'Offer Size' FLOAT DECIMAL (0),
        "bidPrice" 'Bid Price' FLOAT DECIMAL (2) RELATIVE TO "offerPrice",
        "bidSize" 'Bid Size' FLOAT DECIMAL (0)
    );
    CLASS "deltix.timebase.api.messages.TradeMessage" 'Trade Message' UNDER "deltix.timebase.api.messages.MarketMessage" (
        "price" 'Trade Price' FLOAT DECIMAL (2),
        "size" 'Trade Size' FLOAT DECIMAL (0)
    );
)
OPTIONS (DF = 1; HIGHAVAILABILITY = FALSE)
COMMENT 'QQL is awesome'
/
tip

It is not allowed to create a stream with a duplicate key or name, so if you want to try and create a stream like bars, you have to change the key as well.

Create Topic​

info

Create Topic DDL is available starting with TimeBase Enterprise Edition 5.6.121+

Create topic grammar is similar to Create Stream:

CREATE TOPIC [IF NOT EXISTS] topic_name
(class_expression|enum_expression [; ...])
[OPTIONS (identifier [= expr] [; ...])]

Options​

The OPTIONS list below contains all options for identifier:

Click to see stream `OPTIONS` attributes.
  • COPY_STREAM: (Varchar.) Stream name for carbon copy of topic.
  • COPY_SPACE: (Varchar.) Space name for stream copy of topic.
  • MEDIA_TYPE: (Varchar, values: IPC, MULTICAST, UDP_SINGLE_PUBLISHER) Type of topic.
  • PUBLISHER_ADDRESS: (Varchar.) IP address or hostname of the host that will run publisher. Optionally with port (like "10.15.100.1:5555").
  • BUFFER_LENGTH: (Numeric.) Term buffer length for topic.
  • ENDPOINT_HOST: (Varchar.) Multicast group IP address or a hostname that resolves to a multicast IP.
  • ENDPOINT_PORT: (Numeric.) Port for multicast
  • NETWORK_INTERFACE: (Varchar.) Network interface for sending messages (from publisher) and receiving messages (by consumer)
  • TTL: (Numeric.) TTL for multicast packets.

Modify a Stream​

Syntax​

The following grammar is used to modify a stream: 

MODIFY STREAM stream_name [title]
(class_expr|enum_expr [; ...])
[OPTIONS (identifier [= expr] [; ...])]
[COMMENT 'comment text']
[CONFIRM NO_CONVERSION|CONVERT_DATA|DROP_ATTRIBUTES|DROP_TYPES|DROP_DATA]

class_expression is described as follows:  

CLASS type_name [title] [UNDER type_name]
(static_attribute|attribute [, ...])
[AUXILIARY|NOT AUXILIARY]
[INSTANTIABLE|NOT INSTANTIABLE]
[COMMENT 'comment text']

enum_expression is described as follows:  

ENUM enum_name [title]
(identifier [= expr] [, ...])
[FLAGS]
[COMMENT 'comment text']

static_attribute has the following grammar:  

STATIC identifier [title] type [NOT NULL] [encoding] [BETWEEN min_expr AND max_expr] = expr
[TAGS (identifier:expr [, ...])]
[COMMENT 'comment text']

attribute has the following grammar:  

identifier [title] type [NOT NULL] [encoding] [BETWEEN min_expr AND max_expr] [RELATIVE TO identifier] [DEFAULT expr]
[TAGS (identifier:expr [, ...])]
[COMMENT 'comment text']

Options​

The OPTIONS list below contains all options for identifier:

Click to see stream `OPTIONS` attributes.
  • FIXEDTYPE: (Boolean.) A stream capable of containing messages of a single specified type.
  • POLYMORPHIC: (Boolean.) A polymorphic stream is capable of containing messages of several specified types.
  • LOSSLESS: (Boolean.) Lossless stream. Durable streams are always lossless.
  • LOSSY: (Boolean.) Lossy stream.
  • HIGHAVAILABILITY: (Boolean.) High availability durable streams are cached on startup.
  • PERIODICITY: (Varchar.) Indicate a known stream periodicity.
  • DF: (Numeric.) Distribution factor value.
  • INITSIZE: (Numeric.) Initial size of the write buffer in bytes.
  • MAXSIZE: (Numeric.) The limit on buffer growth in bytes. Default is 64K.
  • MAXTIME: (Numeric.) The limit on buffer growth as difference between first and last message time. Default is Long.MAX_VALUE.

Identifier​

An identifier is a token that forms a name.

Confirm​

The CONFIRM block indicates what operations are allowed when you want to change the schema:

  • NO_CONVERSION: Data convertion is not allowed and the MODIFY statement will fail (for example, when data type of a field changes from INTEGER to FLOAT).
  • CONVERT_DATA: Any data convertion is allowed.
  • DROP_ATTRIBUTES: Removing fields is allowed.
  • DROP_TYPES: Removing types is allowed.
  • DROP_DATA: Allow removing data that cannot be converted (for example, when the data type changes from VARCHAR to FLOAT).

Tips​

  • Use UNDER to indicate a parent class if applicable.
  • Classes may include static and/or non-static attributes. You can list them separated by a comma.
  • AUXILIARY classes cannot be written to a stream, but stream objects may contain AUXILIARY classes. NOT AUXILIARY classes can be written to a stream.
  • NOT INSTANTIABLE are abstract classes.
  • Use FLAGS to store enum values encoded as bitmask.
  • Supply a specific expr values with STATIC attributes.
  • Use BETWEEN/END to assign minimal and maximal values to numeric attributes.
  • Non-static attributes may acquire DEFAULT values.
  • Use RELATIVE TO to indicate that a non-static attribute's decoding depends on another field/property value.
  • With TAGS you can store a specific field-related metadata as a key:value pair.
  • Use COMMENT to add your message to the query.

For a general overview, refer to the QQL Introduction page.

Example​

MODIFY STREAM TEST (
    CLASS "deltix.timebase.api.messages.MarketMessage" 'Market Message' (
        "currencyCode" 'Currency Code' INTEGER SIGNED (16) COMMENT 'Currency code represented as short',
        "originalTimestamp" 'Original Timestamp' TIMESTAMP COMMENT 'Exchange Time is measured in milliseconds that passed since January 1, 1970 UTC',
        "sequenceNumber" 'Sequence Number' INTEGER COMMENT 'Market specific identifier of the given event in a sequence of market events',
        "sourceId" 'Source Id' VARCHAR ALPHANUMERIC (10) COMMENT 'Identifies market data source' 
    ) AUXILIARY;
      
    CLASS "deltix.timebase.api.messages.BestBidOfferMessage" 'Quote Message' UNDER "deltix.timebase.api.messages.MarketMessage" (
        "offerPrice" 'Offer Price' FLOAT DECIMAL (2),
        "offerSize" 'Offer Size' FLOAT DECIMAL (0),
        "bidPrice" 'Bid Price' FLOAT DECIMAL (2) RELATIVE TO "offerPrice",
        "bidSize" 'Bid Size' FLOAT DECIMAL (0)
        "exchangeId" 'Exchange' VARCHAR ALPHANUMERIC (10) COMMENT 'Exchange Code',
    );
    CLASS "deltix.timebase.api.messages.TradeMessage" 'Trade Message' UNDER "deltix.timebase.api.messages.MarketMessage" (
        "price" 'Trade Price' FLOAT DECIMAL (2),
        "size" 'Trade Size' FLOAT DECIMAL (0)
    );
)
OPTIONS (DF = 1; HIGHAVAILABILITY = FALSE)
COMMENT 'QQL is awesome'
/

Drop a Stream​

The following grammar is used:

DROP STREAM [IF EXISTS] stream_name

For example:

DROP STREAM test
tip

You will notice that the capitalization of the stream key is not important. That is because, in all cases, QQL searches for existing identifiers in a case-insensitive manner, as long as they are unambiguous.

Drop Topic​

info

Drop Topic DDL is available starting with TimeBase Enterprise Edition 5.6.121+

The following grammar is used:

DROP TOPIC [IF EXISTS] topic_name